Skip to content

Export: trimming enhancements#3530

Open
DuBento wants to merge 78 commits into
thought-machine:masterfrom
DuBento:export-from-package-metadata
Open

Export: trimming enhancements#3530
DuBento wants to merge 78 commits into
thought-machine:masterfrom
DuBento:export-from-package-metadata

Conversation

@DuBento

@DuBento DuBento commented May 1, 2026

Copy link
Copy Markdown
Contributor

Enhancements to plz export, moving from a basic target-level trimming (using gc.RewriteFile) to build statement-level trimming, including only the required build rules and subincludes.
For consistency, we format all the exported BUILD files.

Changelog:

  • Introduced PackageMetadata to track the relationship between BUILD file statements, the targets they generate, and the subincludes they require. Made optional to avoid the overhead for most ops and enabled for the export.
  • Introduced ScopeMetadata to track object origins (subincluded labels) and statement tracking during the interpreter phase. Made optional to avoid the overhead for most ops and enabled for the export.
  • Refactored src/export/export.go to enforce better separation of the DefaultExporter (for trimming) and NoTrimExporter.
  • Added logic to parse BUILD files and selectively write back statements based on whether the generated targets are part of the export set.
  • Implemented "minimal subinclude" generation, which rewrites subinclude() calls to only include labels actually used by the exported targets.
  • Update and added some of the e2e to reflect the changes in implementation.
  • Moved trimming from a GC-based target removal to a full AST statement walk.

duarte added 30 commits April 29, 2026 19:28
setting subincludes at package level instead of at target level
- register subinclude statements in the package metadata
- filter subincludes label
- export all non build_target related statements
…dary build def with sources

the updated test uses non-standard child naming to validate new trimming logic
DuBento added 23 commits May 12, 2026 17:53
- export and trim for stmts: implementation and tests
- remove unused indent level
- trim if-else stmt by keeping clauses but writing a pass those not required
- rework export to trim per stmt and include extra parsing info for if-else stmt
reworked activeSubincludes compute by relying on new logic
@DuBento DuBento changed the title Export: enhance trimming using build statement metadata Export: trimming enhancements Jun 5, 2026
Comment thread src/core/package_metadata.go
Comment thread src/core/package_metadata.go
Comment thread src/core/package_metadata.go
Comment thread src/core/package_metadata.go
Comment thread src/core/package_metadata.go
return func() *core.BuildStatement {
stmtScope := s
for curr := s; curr != nil; curr = curr.callerScope {
if curr.pkg != nil && curr.filename == s.pkg.Filename {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your test for CurrentBuildStatement is fine, as it's relatively understandable what the test data represents. I'm less sure about the ActiveSubincludes test, which has a lot more logic and it's less clear what's going on.

In general, I'm more in favour of setting up a test repo with BUILD files/defs which actually get parsed, as that's much closer to the public interface to the interpreter/parser. Injecting some custom native code for testing actually seems like quite an elegant solution to me - it would effectively be some sort of assertion function?

Comment on lines +831 to +832
standaloneScope := &scope{metadata: newScopeMetadata()}
standaloneScope.metadata.SetCursor(rootStmt)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, let's move this out of the subtest

Comment thread src/plz/plz.go Outdated
if state.Cache != nil {
state.Cache.Shutdown()
}
if state.RemoteClient != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, why has this moved?

Comment thread src/plz/plz.go Outdated
}

// CleanUp cleans up and shuts down the build state.
func CleanUp(state *core.BuildState) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: CleanUp -> Cleanup

Comment thread src/please.go
Test bool
IsQuery bool
ParseMetadata bool
// Keep the workers running in the background for inline parsing during specific ops (e.g. export).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ops -> operations

Similarly op below. No need to abbreviate comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants